Robust Estimation of 3D Human Poses from a Single Image

Human pose estimation is a key step to action recognition. We propose a method of estimating 3D human poses from a single image, which works in conjunction with an existing 2D pose/joint detector. 3D pose estimation is challenging because multiple 3D poses may correspond to the same 2D pose after projection due to the lack of depth information. Moreover, current 2D pose estimators are usually inaccurate which may cause errors in the 3D estimation. We address the challenges in three ways: (i) We represent a 3D pose as a linear combination of a sparse set of bases learned from 3D human skeletons. (ii) We enforce limb length constraints to eliminate anthropomorphically implausible skeletons. (iii) We estimate a 3D pose by minimizing the $L_1$-norm error between the projection of the 3D pose and the corresponding 2D detection. The $L_1$-norm loss term is robust to inaccurate 2D joint estimations. We use the alternating direction method (ADM) to solve the optimization problem efficiently. Our approach outperforms the state-of-the-arts on three benchmark datasets

Towards Reliable Image Outpainting: Learning Structure-Aware Multimodal Fusion with Depth Guidance

Image outpainting technology generates visually plausible content regardless of authenticity, making it unreliable to be applied in practice. Thus, we propose a reliable image outpainting task, introducing the sparse depth from LiDARs to extrapolate authentic RGB scenes. The large field view of LiDARs allows it to serve for data enhancement and further multimodal tasks. Concretely, we propose a Depth-Guided Outpainting Network to model different feature representations of two modalities and learn the structure-aware cross-modal fusion. And two components are designed: 1) The Multimodal Learning Module produces unique depth and RGB feature representations from the perspectives of different modal characteristics. 2) The Depth Guidance Fusion Module leverages the complete depth modality to guide the establishment of RGB contents by progressive multimodal feature fusion. Furthermore, we specially design an additional constraint strategy consisting of Cross-modal Loss and Edge Loss to enhance ambiguous contours and expedite reliable content generation. Extensive experiments on KITTI and Waymo datasets demonstrate our superiority over the state-of-the-art method, quantitatively and qualitatively

Over-expression of an S-domain receptor-like kinase extracellular domain improves panicle architecture and grain yield in rice.

The S-domain receptor kinase (SRK) comprises a highly polymorphic subfamily of receptor-like kinases (RLKs) originally found to be involved in the self-incompatibility response in Brassica. Although several members have been identified to play roles in developmental control and disease responses, the correlation between SRKs and yield components in rice is still unclear. The utility of transgenic expression of a dominant negative form of SRK, OsLSK1 (Large spike S-domain receptor like Kinase 1), is reported here for the improvement of grain yield components in rice. OsLSK1 was highly expressed in nodes of rice and is a plasma membrane protein. The expression of OsLSK1 responded to the exogenous application of growth hormones, to abiotic stresses, and its extracellular domain could form homodimers or heterodimers with other related SRKs. Over-expression of a truncated version of OsLSK1 (including the extracellular and transmembrane domain of OsLSK1 without the intracellular kinase domain) increased plant height and improve yield components, including primary branches per panicle and grains per primary branch, resulting in about a 55.8% increase of the total grain yield per plot (10 plants). Transcriptional analysis indicated that several key genes involved in the GA biosynthetic and signalling pathway were up-regulated in transgenic plants. However, full-length cDNA over-expression and RNAi of OsLSK1 transgenic plants did not exhibit a detectable visual phenotype and possible reasons for this were discussed. These results indicate that OsLSK1 may act redundantly with its homologues to affect yield traits in rice and manipulation of OsLSK1 by the dominant negative method is a practicable strategy to improve grain yield in rice and other crops

Research on active control strategy of vibration in complex environment

FxLMS algorithm has been widely used in active vibration control field theoretically. This paper is aimed at the complex situations in actual environment including interference and occasional divergence due to algorithm. Firstly the effects to control process and result caused by those situations are analyzed, then select different means based on different characteristics of the effects to deal with them, and integrate all those means to derive a new optimal control strategy which is suitable to actual applications. The experiment shows that the improved control strategy can response effectively different occasional situations without any weakness of normal control, and it can promote the practical application ability of the algorithm and is able to adapt to complex environments in active vibration control

Deep Rectangling for Image Stitching: A Learning Baseline

Stitched images provide a wide field-of-view (FoV) but suffer from unpleasant irregular boundaries. To deal with this problem, existing image rectangling methods devote to searching an initial mesh and optimizing a target mesh to form the mesh deformation in two stages. Then rectangular images can be generated by warping stitched images. However, these solutions only work for images with rich linear structures, leading to noticeable distortions for portraits and landscapes with non-linear objects. In this paper, we address these issues by proposing the first deep learning solution to image rectangling. Concretely, we predefine a rigid target mesh and only estimate an initial mesh to form the mesh deformation, contributing to a compact one-stage solution. The initial mesh is predicted using a fully convolutional network with a residual progressive regression strategy. To obtain results with high content fidelity, a comprehensive objective function is proposed to simultaneously encourage the boundary rectangular, mesh shape-preserving, and content perceptually natural. Besides, we build the first image stitching rectangling dataset with a large diversity in irregular boundaries and scenes. Experiments demonstrate our superiority over traditional methods both quantitatively and qualitatively.Comment: Accepted by CVPR2022 (oral); Codes and dataset: https://github.com/nie-lang/DeepRectanglin

Neural Contourlet Network for Monocular 360 Depth Estimation

For a monocular 360 image, depth estimation is a challenging because the distortion increases along the latitude. To perceive the distortion, existing methods devote to designing a deep and complex network architecture. In this paper, we provide a new perspective that constructs an interpretable and sparse representation for a 360 image. Considering the importance of the geometric structure in depth estimation, we utilize the contourlet transform to capture an explicit geometric cue in the spectral domain and integrate it with an implicit cue in the spatial domain. Specifically, we propose a neural contourlet network consisting of a convolutional neural network and a contourlet transform branch. In the encoder stage, we design a spatial-spectral fusion module to effectively fuse two types of cues. Contrary to the encoder, we employ the inverse contourlet transform with learned low-pass subbands and band-pass directional subbands to compose the depth in the decoder. Experiments on the three popular panoramic image datasets demonstrate that the proposed approach outperforms the state-of-the-art schemes with faster convergence. Code is available at https://github.com/zhijieshen-bjtu/Neural-Contourlet-Network-for-MODE.Comment: IEEE Transactions on Circuits and Systems for Video Technolog